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An Investigation of the Relationship between In-Class 
and Out-of-Class Efforts on Student Learning: Empirical 
Evidence and Strategy Suggestion 

Tin-Chun Lin 1 

Abstract: In this paper we explore and discuss an important research question in 
higher education - is there a trade-off relationship between in-class and out-of- 
class efforts for students? We used an empirical model to test the trade-off 
hypothesis between these two efforts. We identified a trade-off between in-class and 
out-of-class efforts, especially for those students who do not perform well on 
examinations. We clarified possible reasons for this relationship in a lower- 
performing student group and noted potentially harmful implications for higher 
education. We recommended that instructors work individually with students in 
setting appropriate gocds for each exam and frequently offering feedback. Doing 
so can strengthen rapport between students and faculty, thereby enhancing 
students ’ motivation to learn and confidence in utilizing faculty as a learning 
resource. We also recommended a classroom-based gameplay strategy to promote 
students ’ motivation to learn and encourage their participation. 

Keywords: in-class effort; out-of-class effort; complements; substitutes; goal 
setting theory; Rapport. 

Introduction 

In addition to an instructor’s teaching skills and a student’s quality, both in-class effort and out- 
of-class effort are two primary factors in determining the extent of a student’s learning outcome. 
Since these two major factors influence a student’s acquisition of knowledge, we wondered what 
the relationship would be between them. We normally would expect students to jointly employ 
these two factors to pursue their knowledge—students not only frequently attend classes but also 
study regularly outside the classroom. The reason is that students receive information via the 
instructor’s classroom lecture but need time to absorb and process it outside the classroom so that 
it can become their own knowledge. Nevertheless, a question raised here is: for some students, 
would it be possible for these two factors to trade off? Some students frequently skip classes but 
may study hard outside the classroom. For that reason, is it possible that these two factors may 
substitute for each other for these students? 

Acknowledging that students are a diverse group, it is possible for in-class effort and out- 
of-class effort to be complementary for some students while substituting for each other among 
other students. Thus, the overall effect will depend on which effect is dominant. The substitution 
of in-class effort and out-of-class effort during the development of knowledge may mean one of 
two things: (1) the instructor may lecture very well—clear, organized, and understandable such 
that relative to in-class effort, students do not need to devote much out-of-class effort to studying 
for the course; or (2) students may not understand the instructor’s lecture mainly due to his/her 
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poor instructional skills or inability to match students’ needs for learning—hence, students may 
prefer to study by themselves outside the classroom rather than devoting more effort inside the 
classroom. 

In addition, can gender be a factor in determining the relationship? According to findings 
from a previous study by Woodfield, Jessop, and McMillan (2006), male students missed classes 
more frequently than did female students. This evidence inspires us to reasonably suspect that in- 
class effort and out-of-class effort for male students may possibly substitute for each other. Our 
logic could be wrong, but the idea may be described as follows: when students who frequently 
skip a class still successfully pass the course, there is a possibility that they may work harder 
outside the classroom. In other words, these students’ in-class and out-of-class efforts may 
substitute for each other. Moreover, according to data used for this study (see Table 5), lower- 
performing students are more likely to be absent from classes than higher-performing students. 
While there are many reasons for being absent, lower-performing students may become frustrated, 
lose their trust and confidence in using faculty as a learning resource in the classroom, and elect 
to study by themselves outside the classroom rather than sitting inside the classroom. Therefore, 
we also reasonably suspect that in-class effort and out-of-class effort may likely substitute for each 
other among lower-performing students. To determine whether or not our thoughts may be correct, 
we further tested these two hypotheses. 

Three research questions were developed for this study: (1) Are in-class effort and out-of- 
class effort related? If yes, is the relationship based on complements or substitutes? (2) If the 
relationship is based on substitutes, who would most likely exhibit such behavior—higher- 
performing or lower-performing students? (3) Can gender be a factor in determining whether in- 
class and out-of-class efforts may substitute for each other? In addition to these three research 
questions we discuss three implied questions: (1) why would these two efforts trade off each other? 
(2) What would be the impact on higher education if these two efforts trade off each other? (3) 
What methods could be suggested to educators to assist students (especially lower-performing 
students) in achieving success? 

Although this topic may be related to several researchers’ work on the relationship between 
students’ exam performance and lecture attendance (e.g., Anikeeff, 1954; Jones, 1984; Brocato, 
1989; Park and Kerr, 1990; Van Blerkom, 1992; Gunn, 1993; Romer, 1993; Day, 1994; Durden 
and Ellis, 1995; Rodgers, 2001; Chung, 2004; Stanca, 2006; and Dolnicar, Kaiser, Matus, and 
Vialle, 2009), we focused on a different perspective—the relationship between in-class effort and 
out-of-class effort. We used the complementarity test to investigate the trade-off hypothesis 
between these two efforts. Our study findings will provide educators with a closer look at the 
relationship between students’ in-class effort and out-of-class effort, offering new ways to 
understand student behavior and to assist them in achieving success. 

In this paper we not only provide a detailed discussion of the results from our investigation 
of the relationship between in-class effort and out-of-class effort, but also offer a thoughtful 
discussion of why a relationship between these two efforts would arise and how instructors may 
assist students in attaining success. In addition, we divide data according to performance (i.e., 
higher-performing and lower-performing) and gender (i.e., male and female) to ascertain whether 
these groups behave differently—whether or not the relationship between in-class effort and out- 
of-class effort serves as a substitute, which will be an important result as we seek to understand 
why some students skip classes more frequently, and what we can do about it, while others do not. 
More importantly, this paper extends the literature on the issue of student effort-investment and 
academic achievement in higher education. 
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This paper makes a number of contributions to the higher education literature, because this 
study offers the first descriptive look at the use of the complementarity test to investigate this issue, 
linking the relationship between in-class and out-of-class efforts on student learning. In other 
words, this paper is beneficial to both faculty teaching and student learning in higher education. 

A Brief Literature Review 

While the relationship between in-class effort and out-of-class effort has not yet been investigated 
and discussed, it is related to the topics of motivation, effort, and academic performance. 
Motivation is like an engine that moves learning goals forward (Rothestein, 1990; Woolfolk, 1990). 
Without motivation, no matter how smart the student is, he or she will not learn anything. Students 
with differing motivations for enrolling in the same class will experience different levels of 
academic achievement (Nicholls, 1984; Dweck, 1986). That is, a student’s motivation determines 
his/her effort-investment and hence academic performance. More clearly, study strategies (Elliott, 
McGregor, and Gable, 1999) and quality of effort (Plant, Ericsson, and Asberg, 2005) are 
important to a student’s learning when time boosts learning. These studies indicate that quality of 
effort is determined by motivation. Below, we briefly review the literature in these three topics. 

The topic of motivation has been broadly studied and defined by psychologists such as 
Maslow (1954), Aderman (1999), Murray, Poole, and Jones (2006), and Vansteenkiste et al. 
(2009). Motivation is an important concept in psychology that offers insight into why we may 
behave the way we do. It is an internal process that reflects our desire to achieve certain goals, 
such as academic achievement. Goal achievement requires motivation and more—a mediator is 
also needed, such as effort (classroom attendance, and the amount of time devoted to studying and 
completion of assignments) to achieve goals. Therefore, numerous researchers (e.g., Goodman et 
al., 2011; Kusurkar et al., 2013) investigated the relationship between students’ motivation and 
academic performance as mediated by effort. 

Motivation can be divided into two types: intrinsic motivation and extrinsic motivation. 
Findings from Goodman et al. (2011) indicated that both intrinsic and extrinsic motivations are 
positively and significantly associated with students’ academic performance as measured by GPA 
(Grade Point Average). Both types of motivation could positively affect the amount of effort 
invested in academic performance. Thus, effort is an important mediator in linking the relationship 
between intrinsic/extrinsic motivation and academic performance, and intrinsic motivation is an 
even stronger predictor of academic achievement than extrinsic motivation. 

While motivation includes intrinsic and extrinsic motivations, the quality values of 
motivation are quite different. Kusurkar et al. (2013) used self-determination theory to differentiate 
them into two categories: autonomous and controlled. Between these two motivations is a relative 
autonomous motivation, which is a measure of the balance between autonomous motivation and 
controlled motivation. Findings from Kusurkar et al. (2013) demonstrated that relative autonomous 
motivation positively and significantly affected students’ study strategies and efforts and in turn 
influenced their academic performance. 

As discussed earlier, efforts play a very important role as a mediator between motivation 
and academic performance. Efforts include in-class effort (i.e., lecture attendance) and out-of-class 
effort (i.e., the amount of time devoted to studying and completion of assignments outside the 
classroom). Several researchers (e.g., Jones, 1984; Rodgers, 2001; Cohn and Johnson, 2006; 
Kember et al., 1995; Zulauf and Gortner, 1999) have mainly focused on the effects of efforts (either 
in-class or out-of-class effort) on academic performance. 
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The relationship between attendance (in-class effort) and grades can be indirect or direct 
(Jones, 1984). Since students have a motivation to earn a college degree, they will attend class, 
implying that attendance and grades may be indirectly related. However, attendance and grades 
can also be directly related; that is, more frequent attendance may lead to higher grades. Lower 
grades will discourage students from attending class; thus, attendance and grades will be directly 
related. In addition, heterogeneous variables, such as intelligence and motivation, among students 
might influence students’ attending behavior and hence affect their performance (Rodgers, 2001). 
Moreover, the effect of all levels of attendance on student performance is not identical—low 
performance among non-attendees is particularly significant among those missing a large fraction 
of classes (Cohn and Johnson, 2006) 

In addition to in-class effort, researchers such as Kember et al. (1995) and Zulauf and 
Gortner (1999) focused on the impact of out-of-class effort on academic performance. Kember et 
al. (1995) found that whether students would spend more or less time on studying outside the 
classroom depended on their learning approach. Students using a surface approach had a higher 
lecture attendance rate and spent more time on studying and hence received higher grades. A deep 
approach showed that students did not receive higher grades unless they offered sufficient work. 
Findings from Zulauf and Gortner (1999) demonstrated that the amount of time spent studying 
outside the classroom exerted a positive and significant effect on students’ academic performance 
as measured by GPA (grade point average). They also found that the time management score had 
a positive impact on GPA, implying that better time management skills could significantly improve 
students’ academic achievement. 

Above all, we learned that previous researchers had either studied the relationship between 
students’ motivation and academic performance as mediated by effort or the relationship between 
effort (in-class or out-of-class effort) and academic performance. In this study, we investigated the 
relationship between in-class effort and out-of-class effort. In the following sections, we show our 
detailed investigation and discuss our findings. 

Method 

Data 

The data used in this study are the same as the data used in Lin (2014). Please refer to Lin (2014) 
for a detailed description of the data and experimental design. However, we still briefly illustrate 
the data and variables used in this study below. 

Total participants in this study were 389 business students in Introduction to 
Microeconomics classes in spring 2007, 2009, and 2011, and fall 2012. These students were taught 
by the same instructor. The data sources include instructor-reported data, university-supplied data, 
and student-self-reported data (i.e., survey response). The instructor-reported data consist of 
students’ attendance record (i.e., in-class effort) and three exam scores 2 (i.e., learning outcomes); 
while the university-supplied data contain students’ GPA (grade point average) and SAT 
(Scholastic Aptitude Test) scores. In addition, the student-self-reported data included students’ 
work hours for pay per week and out-of-class effort devoted to studying for the class during each 
exam. In a questionnaire, students were asked: (1) how many hours for pay do you work a week? 
(This question was only asked once in the first exam.) (2) Overall, approximately how long did 
you study for the class during this exam period? (This question was asked in each exam.) There 


2 Each exam included multiple-choice and critical analysis essays and problems. 
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were five choices for the question: (i) I study 1-5 hours before the test; (ii) I study 6-10 hours 
before the test; (iii) I study 11-15 hours before the test; (iv) I study 16-20 hours before the test; 
and (v) I study more than 20 hours before the test. 

It should be noted that the variables of student-quality index (i.e., a combination of GPA 
and SAT scores) and work hours for pay per week were not used in the empirical model, but they 
were used for the discussion. Moreover, we used both GPA and SAT scores to create a variable of 
student-quality index. The GPA, regardless of a student’s major, is a measure of a student’s 
motivation and academic ability 3 ; while SAT scores are a measure of a student’s innate ability 4 . 
To create student quality index, we first combined these two elements to measure a student’s 
quality value. A simple way to combine these two is to use a multiplicative function of GPA and 
SAT score (i.e., a student quality value {QUA) = SAT xGPA) 5 . We then converted QUA values 

into an index that may be described as follows: QUI . = —^—xlOO, where QUT - student 

max QUA 

fs QUA index, QUA f = student fs initial QUA value, and max QUA = the maximum QUA value 
among all students. 6 

Moreover, descriptive statistics (means and standard deviations) for variables used in this 
paper are presented in Table 1. In addition, Cronbach’s alpha was 0.84, indicating strong internal 
consistency among these exams. 


Table 1: Means and Standard Deviations of Variables _ 

Variables Mean Standard Deviation 


3 Karsenti and Thibert (1995) demonstrated that academic motivation and GPA are positively and significantly 
correlated. In 2010, Afzal, Ali, Khan, and Hamid also found that those with high intrinsic motivation will achieve a 
higher GPA. Bacon and Bean (2006) even pointed out that GPA explains lots of the variance in learning outcomes/final 
exam scores. 

4 Frey and Detterman (2004) showed that the SAT is an adequate measure of general intelligence. In 2008, Koening, 
Frey, and Detterman further demonstrated that the ACT (American College Test) and SAT are positively and 
significantly correlated, and ACT scores can be used to accurately predict a student’s IQ (Intelligence Quotient), 
implying that SAT scores also can be an accurate predictor for a student’s IQ. Moreover, SAT has been widely used 
by researchers as a proxy for student ability (e.g., Lin and Quayes, 2006; Dynarski and Gleason, 1993). 

5 The multiplicative method has been used by researchers (e.g., Lin [2010] measured student achievement as a 
multiplicative function of SAT score and the proportion of public school graduates enrolled in 2- or 4-year colleges 
and universities). 

6 The idea of combining motivation (GPA) and ability (SAT scores) into one measure (quality index) comes from the 
meaning of motivation expressed by psychologists (e.g., Rothestein, 1990; Woolfolk, 1990). Let’s take a look at the 
example of a car. If a car’s engine is low quality, no matter how beautiful and solid the car looks, the car cannot run 
fast and far and hence cannot be considered a high-quality car. Similarly, even though a student is smart, if the student 
has very low or zero motivation to learn the course, the student will not learn anything. Thus, the student cannot be 
considered a high-quality student. Therefore, the advantage of combining motivation and ability into one measure is 
to effectively reflect a student’s “real” quality values. 
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Scores for exam I 

Scores for exam II 

Scores for exam III 

SAT scores 

Grade Point Average 

Work hours for pay per week 

Frequency of studying for 1 st exam 

Frequency of studying for 2 nd exam 

Frequency of studying for 3 rd exam 

Attendance in the 1 st exam period 

Attendance in the 2 nd exam period 

Attendance in the 3 rd exam period 


69.24 

17.63 

77.91 

18.81 

56.49 

18.03 

973.28 

127.49 

2.73 

0.58 

28.56 

13.14 

2.74 

1.11 

3.23 

1.17 

3.02 

1.24 

9.09 

1.22 

8.49 

1.82 

8.55 

1.96 


Econometric Model 

The complementarity test has been broadly used in the economics literature to examine the 
relationship between labor and capital. Thus, we applied the complementarity formation used by 
economics researchers (e.g., Lin, 2003) to investigate the trade-off hypothesis between in-class 
and out-of-class efforts. The empirical model (i.e., the complementarity formation) is shown as 
follows: 

In G t — In G t _ x = D 0 + a A [in A, - In A t _ 1 ]+ a L [in L t - In L tX ] + a M [(in A [ f ~ (h A-i f\/ 2 

+ a LL [(in L t ) 2 - (in L t _ x ) 2 J/2 + [in A In L t - In A r _ x In L t _ x ] + u t , (1) 

where t = the current exam period; t— 1 = the last exam period; G = exam scores (i.e., learning 
outcomes), A = attendance (i.e., in-class effort), L = frequency of studying for an exam (i.e., out- 
of-class effort), u t = stochastic disturbance terms assuming a mean 0 and a variance a 1 . If in- 

class effort and out-of-class effort are substitutes, a AL < 0 and the effect should be significant. 

Results 

In this section we present our empirical evidence from Equation (1) for three hypotheses based 
upon our three research questions. Our presentation focuses on the coefficient ( a Al ), which is the 
most important result needed here. 

Hypothesis 1 : In-class effort and out-of-class effort are related, while the relationship can be 
complements or substitutes. 

For Hypothesis 1, the data included all 389 students. The results for Equation (1) for all 
students arc reported in Table 2. In the earlier periods (Periods 1-2), as Column (1) of Table 2 
shows, the coefficient ( a Af ) was not statistically and significantly different from zero at any level, 
implying that these two efforts are not related. However, in the later periods (Periods 2-3), as 
shown in Column (2) of Table 2, the coefficient ( a AI ) was negative and was statistically and 
significantly different from zero at the 10% level, implying that these two efforts are related and 
may be trading off each other. We now combine all three periods. As Column (3) of Table 2 shows, 
the coefficient (a A ,) was also negative and was statistically and significantly different from zero 
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at the 1% level. This was because the effect of the later periods may dominate the effect of the 
earlier periods. The overall result implies that in-class effort and out-of-class effort are 
significantly related and the relationship may be substitutes, implying that in-class and out-of-class 
efforts may trade off for some (or many) students. 

Table 2; Results for Hypothesis 1 - All Students _ 

Explained Variable: InG, —lnG M 


Explanatory Variables 

(1) 

Periods 1-2 

(2) 

Periods 2-3 

(3) 

All Periods 

Constant 

0.046*** 

-0.296*** 

0 1/]/]*** 

In A, - In A t l 

(3.54) 

-0.253** 

(-23.89) 

0.015 

(-14.59) 

0.036 

In L t - In L t , 

(-2.00) 

0.341** 

(0.10) 

0.651*** 

(0.31) 

0.823*** 

(hi A,) 2 - (in A tl ) 2 

(2.05) 

0.218*** 

(4.05) 

0.143 

(6.18) 

0.062 

2 

(2.74) 

(1.42) 

(0.83) 

(in L t ) 2 - (in L r j ) 2 

0.198** 

0.160 

0.452*** 

2 

(2.27) 

(1.38) 

(5.34) 

In A, In L, -In A,_j In L r _, 

-0.031 

-0.143* 

-0.251*** 

R 2 

(-0.41) 

0.365 

(-1.71) 

0.348 

(-3.96) 

0.438 

R 2 

0.357 

0.340 

0.435 

F- Statistics 

44.06 

40.94 

120.55 

Observations 

389 

389 

778 


Note: Number in parentheses is f-value. All periods are the combination of Periods 1-2 and Periods 2-3, so the 
observations are double of the initial observations. 

***/?< .01; **p< .05; *p< .10 

Consequently, overall, we had sufficient evidence to support Hypothesis 1—in-class and 
out-of-class efforts are significantly related for many students, and the relationship may be 
substitutes. The results for Hypothesis 1 leave two questions: (1) who would most likely exhibit 
such behavior—higher-performing or lower-performing students? (2) Why would the trade-off 
relationship only occur in the later periods (i.e., Periods 2-3) rather than in the earlier periods (i.e., 
Periods 1-2) or both? The first question is answered in the results for Hypothesis 2; while the 
second question is examined in the discussion section. 

Hypothesis 2 : The trade-off relationship between in-class effort and out-of-class effort would most 
likely occur in the lower-performing student group. 

In hypothesis 2, we split data according to higher-performing and lower-performing 
students based on their final course grades. We defined higher-performing students as students 
who received A or B grades for the course, while lower-performing students received C or lower 
grades (i.e., D and F grades) for the course. The results for Equation (1) for higher-performing and 
lower-performing students are reported in Table 3. In the earlier periods (Periods 1-2), as shown 
in Columns (1) and (2) of Table 3, the coefficient ( a A! ) is not statistically and significantly 
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different from zero at any level for both the higher-performing student group and the lower- 
performing student group. In the later periods (Periods 2-3), while the coefficient ( a AI ) is still not 
statistically and significantly different from zero at any levels for the higher-performing student 
group, the coefficient () is negative and statistically and significantly different from zero at the 
10% level for the lower-performing student group. We then combined all three periods. As 
Columns (5) and (6) of Table 3 show, the coefficient ( a AI ) for the higher-performing student group 
is still not statistically and significantly different from zero at any level, but the coefficient ( a A] ) 
for the lower-performing student group is negative and statistically and significantly different from 
zero at the 1% level. This is also because the effect of the later periods may dominate the effect of 
the earlier periods. The results imply that no remarkable relationship exists between in-class effort 
and out-of-class effort in the higher-performing student group; however, overall, the relationship 
between in-class effort and out-of-class effort may substitute for one another in the lower- 
performing student group. 

Table 3: Results for Hypothesis 2 - Higher- and Lower-Performing Students _ 

Explained Variable: In G, - In G r _, 

Periods 1-2 Periods 2-3 All Periods 



(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Explanatory 

Higher- 

Lower- 

Higher- 

Lower- 

Higher- 

Lower- 

Perform 

Perform 

Perform 

Perform 

Perform 

Perform 

Variables 

Students 

Students 

Students 

Students 

Students 

Students 

Constant 

0.03** 

0.06*** 

-0.26*** 

-0.33*** 

-0.13*** 

-0.15*** 


(2.54) 

(2.90) 

(-18.79) 

(-17.54) 

(-11.31) 

(-10.31) 

In A, -lnA f _j 

0.43 

-0.26* 

0.22 

0.03 

0.79** 

0.03 

(0.70) 

(-1.67) 

(0.55) 

(0.19) 

(2.06) 

(0.24) 

In L ( -In L ( l 

0.52 

0.28 

0.68 

0.52** 

0.70 

0.79 

(1.17) 

(1.32) 

(1.37) 

(2.49) 

(1.61) 

(4.75) 

On A)' “On A ,) 2 

2 

0.03 

0.22 

0.15 

0.11 

-0.27 

0.05 

(0.09) 

(2.22) 

(0.56) 

(0.92) 

(-1.11) 

(0.53) 

(in L, ) 2 - (in L, J 

0.30*** 

0.33** 

0.16 

0.31* 

0.38*** 

0.60*** 

2 

(2.85) 

(2.46) 

(0.96) 

(1.79) 

(3.17) 

(4.71) 

In A, In L, - In A^_ x In L r _ x 

-0.20 

-0.02 

-0.23 

-0.11* 

-0.21 

-0 27*** 


(-1.01) 

(-0.19) 

(-0.97) 

(-1.70) 

(-1.11) 

(-3.36) 

R 2 

0.469 

0.368 

0.282 

0.373 

0.466 

0.439 

R 2 

0.452 

0.354 

0.259 

0.359 

0.457 

0.433 

F-Statistics 

27.54 

25.79 

12.26 

26.26 

55.43 

70.24 

Observations 

162 

227 

162 

227 

324 

454 


Note: Number in parentheses is /-value. All periods are the combination of Periods 1-2 and Periods 2-3, so the 
observations are double of the initial observations. 

***p< .01; **p< .05; *p< .10 


In short, overall, we have sufficient evidence to support Hypothesis 2. The evidence shows 
that the trade-off relationship between in-class effort and out-of-class effort would most likely 
occur in the lower-performing student group. The results of Hypothesis 2 also raise the same 
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question as for Hypothesis 1: why would the trade-off relationship occur for lower-performing 
students in the later periods (i.e., Periods 2-3) rather than in the earlier periods (i.e., Periods 1-2) 
or both? We look at this question in the discussion section. 

Hypothesis 3 \ Gender is a factor in determining whether in-class and out-of-class efforts may 
substitute for each other. 

For hypothesis 3, we split data according to gender. The results for Equation (1) for male 
and female students are presented in Table 4. In the both earlier periods (Periods 1-2) and later 
periods (Periods 2-3), as shown in Columns (l)-(4) of Table 4, the coefficient (a^) is not 
statistically and significantly different from zero at any level for both the male student group and 
the female student group. However, when we combine all three periods, according to the results 
shown in Columns (5) and ( 6 ) of Table 4, the coefficient ( a ^) is negative and is statistically and 
significantly different from zero at the 1% level for the male student group and at the 5% level for 
the female student group. 

Table 4: Results for Hypothesis 3 - Male and Female Students _ 

Explained Variable: In G, - In G r _, 

Periods 1-2 Periods 2-3 All Periods 



( 1 ) 

( 2 ) 

(3) 

(4) 

(5) 

( 6 ) 

Explanatory 

Male 

Female 

Male 

Female 

Male 

Female 

Variables 

Students 

Students 

Students 

Students 

Students 

Students 

Constant 

0.01 

0.09*** 

-0 27*** 

-0 32*** 

-0.15*** 

-0.14*** 


(0.46) 

(4.84) 

(-15.06) 

(-18.58) 

(-11.29) 

(-9.26) 

In A, - In A, , 

-0.40* 

-1.19 

0.22 

-0.40 

0.21 

-0.19 


(-1.90) 

(- 1 . 22 ) 

(1.18) 

(-1.51) 

(1.37) 

(-1.05) 

InL, -InL, , 

0.33 

0.37 

0.59*** 

0 79 *** 

0 7g*** 

0.9i*** 


(1.52) 

(1.40) 

(2.59) 

(3.44) 

(4.57) 

(4.34) 

(in A, ) 2 - (in A, , ) 2 

0.28** 

0.23** 

0.002 

0.40** 

-0.04 

0 . 20 * 

2 

(2.31) 

(2.08) 

( 0 . 02 ) 

(2.37) 

(-0.40) 

(1.76) 

(in L ; ) 2 - (in L t J 

0.21 

0.17 

0.27 

0.06 

0.50*** 

0 39 *** 

2 

(1.75) 

(1.30) 

(1.55) 

(0.37) 

(4.40) 

(3.09) 

In A, In L, - In A r-1 In L t _ x 

-0.03 

- 0.02 

-0.15 

-0.15 

-0.26*** 

-0.25** 


(-0.29) 

(-0.24) 

(-1.33) 

(-1.26) 

(-3.19) 

(-2.56) 

R 2 

0.365 

0.395 

0.345 

0.373 

0.447 

0.438 

R 2 

0.349 

0.379 

0.328 

0.357 

0.440 

0.431 

F-Statistics 

22.10 

24.18 

20.25 

22.05 

63.04 

58.70 

Observations 

198 

191 

198 

191 

396 

382 


Note: Number in parentheses is f-value. All periods are the combination of Periods 1-2 and Periods 2-3, so the 
observations are double of the initial observations. 

***/?< .01; **p< .05; *p< .10 


As a result, overall, we do not have sufficient evidence to support Hypothesis 3 because 
the evidence shows that in-class and out-of-class efforts may substitute for each other for both 
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male and female students, implying that gender cannot be a significant factor in this issue. 
Therefore, Hypothesis 3 is rejected. 

Discussion 


The results offer sufficient empirical evidence to support the belief that in-class and out-of-class 
efforts may be trading off each other for some students. The evidence shows that lower-performing 
students are more likely to exhibit such behavior. Therefore, the question here is: why would these 
two efforts trade off each other for lower-performing students in the later periods rather than in the 
earlier periods? Before answering this question, let’s take a quick look at three indicators—quality 
index, attendance rate, and work hours for pay per week for both higher-performing and lower- 
performing students—which could help us to understand the empirical results. As Table 5 shows, 
based upon the one-tailed test (the upper tail test) 7 , both the quality index and attendance rate for 
higher-performing students are statistically and significantly higher than for lower-performing 
students, while lower-performing students’ work hours for pay per week are statistically and 
significantly longer than higher-performing students’. 


Table 5: Results of One-Tailed Test (Upper Tail T est) _ 

Higher- 

Performing 

Students 

(n = 162) 

Variables M SD 

Quality index 61.89 13.78 

Attendance rate 92.90% 9.65% 

Work hours for pay per week _ 23.86 12.52 

Note: M = mean; SD = standard deviation; and n = total observations. 
****Den 0 te S statistical significance of the p-value at the .001 level. 


Lower- 

Performing 

Students 

(n = 227) 

M _ SD P-value 

44.99 13.67 0.000* *** 

82.97% 15.85% 0.000**** 

31.91 12.55 0.000* 


The information on work hours for pay per week may offer an answer. A number of 
researchers have investigated the impact of student employment on school performance. The 
impact can be positive, negative, or unrelated. Several researchers, such as Singh (1998), Eckstein 
and Wolpin (1999), and Oettinger (1999), have shown negative effects of part-time employment 
on school performance; while others have found that academic achievement may improve with 
low work hours but drop with long work hours (e.g., Schill et al., 1985; Lillydahl, 1990; Quirk et 


7 We used the one-tailed test (the upper tail test) by formulating null ( H Q ) and alternative ( H a ) hypotheses: 

[H q . f,i AI H — // v , / ^ 0 [H 0 . /u QI _ H /-Iq; , ^ 0 \H q . f,i WH L — f,i WH H < 0 

\^a • MaT-H ~ MaT-L > 0 \^ a ' ~ Aq/-L > 0 1^0 • I^WH-L ~ MwH-H > ^ 

where jU AT _ H = the population mean for attendance rate in the higher-performing student group; JU AT _ L = the 
population mean for attendance rate in the lower-performing student group; /l ()l H = the population mean for quality 
index in the higher-performing student group; p ()/ 7 = the population mean for the quality index in the lower- 
performing student group; jU WH _ H = the population mean for working hours per week in the higher-performing 
student group; and = the population mean for working hours per week in the lower-performing student group. 
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al., 2001). However, researchers, such as Schoenhals et al. (1998), Warren et al. (2001), and 
Dustmann et al. (2007), did not find a relationship between student employment and school 
performance. 

A related study by DeSimone in 2008 showed that students who worked 30 hours or more 
each week had significantly lower grades. As shown in Table 5, the average number of work hours 
for pay per week for lower-performing students was over 30 hours. No surprisingly, they 
performed poorly because their work hours significantly crowded out their time for academics. 8 
For example, they have less time to study for the class and their employment schedules may 
sometimes conflict with class schedules so that they have to skip classes. Moreover, their quality 
index was lower than that for higher-performing students. Therefore, the only way to improve their 
grades was to reduce work hours and significantly devote their efforts to activities both inside and 
outside the classroom. 

However, things usually do not work as expected. First, it is impossible for students to 
reduce their work hours or quit their jobs because without work, they may not be able to afford 
college. A survey done by King and Bannon in 2002 indicated that 63% of full-time working 
college students admitted that they would discontinue their academic careers if they stopped 
working. Second, lower-performing students may be less motivated, especially when they are not 
interested in the class. More importantly, they probably cannot grasp the lecture immediately. 
Thus, after the midterm, they may become less willing to learn and more reluctant to attend the 
class. If they do not get assistance right away to solve their questions, they probably will not grasp 
the next lecture and will have even more questions. This is because concepts and models taught in 
earlier lectures may be used in later lectures. Not understanding an earlier lecture makes it difficult 
to move on to the next lecture. As questions accumulate, students eventually lose interest and 
choose not to frequently attend future classes. As Table 6 shows, based upon the one-tailed test 
(the lower tail test) 9 , the attendance rate for lower-performing students significantly dropped from 
88.46% in the earlier first exam period to 80.04% in the later third exam period. Since they may 
believe that they cannot catch up or grasp the lecture, it is possible that they may elect to study 
outside the classroom by themselves. Then, when they receive lower grades such as D or F on the 
midterm, they become frustrated and fail to utilize faculty as a learning resource. This further 
decreases their motivation and willingness to learn. Consequently, these students reduce their in- 
class effort while increasing out-of-class effort, resulting in a trade-off relationship between in- 
class and out-of-class efforts after the midterm. 


Table 6: Results of One-Tailed Test (Lower Tail Test) _ 

Exam Period 1 Exam Period 3 
(n = 227 )(n = 227) 


8 It is possible that some student just want the degree, thus perhaps their efforts to work more hours and study less are 
optimal for their goal. 

9 We used the one-tailed test (the lower tail test) by formulating null ( H 0 ) and alternative (H ) hypotheses: 

( H 0 ■ MaT-L3 ~ MaT-L\ — 0 

Ha ■ Mat 13 ~ Mat-li < 0 

where Mat-L i = the population mean for attendance rate in the exam period 3 in the lower-performing student group; 
Mat-l\ = the population mean for attendance rate in the first exam period in the lower-performing student group. 
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Variable M _SD_M_SD_ P-value 

Attendance rate for lower-performing 

students 88.46% 13.20% 80.04% 20.47% 0.000**** 

Note: M = mean; SD = standard deviation; and n = total observations. 

****Den 0 te S statistical significance of the p-value at the .001 level. 

In addition, the evidence implies that many lower-performing students may believe that 
they will perform better if they study outside the classroom and so do not attend the instructors’ 
lectures, since they can find the information in textbooks and/or online. The evidence shows that 
they are not exactly correct since their grades still do not improve after following this plan. Much 
of the information from instructors’ classroom lectures may not be found in textbooks and online. 
Therefore, even though these students may try to study hard outside the classroom, they still may 
not do well on exams. Thus, we may inquire: what would be the impact on higher education if 
these two efforts trade off each other? Based upon the empirical evidence found in this study, the 
trade-off relationship does not seem to have a positive impact on higher education because it may 
result in more students (especially lower-performing students) attempting to skip classes more 
frequently, thereby increasing the gap in performance-inequality values between higher¬ 
performing and lower-performing students. That is, higher-performing students always attend 
classes and receive important information and thus perform well, which encourages them to make 
progress; while lower-performing students frequently skip classes and miss important information 
and hence perform poorly, which discourages their progress. As mentioned earlier, a lot of 
information from classroom lectures may not be found in textbooks and online, which means that 
even if students who skipped a number of classes try to study hard outside the classroom, they may 
not necessarily have access to missed information. Although we have seen some higher¬ 
performing students who frequently skip classes still perform well in that class, we do not 
encourage this behavior. The empirical evidence studied by Lin (2014) demonstrated that even 
higher-performing students’ exam performance progress may be significantly decelerated by 
skipped classes. 

Strategy Suggestion 

Since the trade-off relationship does not seem to have a positive impact on higher education, do 
educators have access to methods that may be used to assist students (especially lower-performing 
students) in achieving success? Students must have many reasons for missing classes, but one 
possible reason for some students’ lower performance is a loss of trust and confidence in 
instructors when they do not understand their lectures. Eventually, these students may choose to 
skip future classes more frequently. In addition, lower-performing students’ reluctance to ask 
questions in class is a common problem. To assist lower-performing students, instructors must be 
very patient and may take the initiative to contact and meet with them. Lower-performing students 
may feel more comfortable asking questions in professors’ offices. That is, educators need to build 
a strong rapport with students in order to improve their learning motivation and ameliorate their 
trust and confidence in using faculty as a learning resource (Granitz, Koerning, and Harich, 2009; 
Young, 2005; Huff, Cooper, and Jones, 2002). Young (2005) provided a detailed discussion about 
enhancing a student’s motivations to learn. As pointed out by Young (2005), an enthusiastic 
teacher should offer students elevated personal interaction with supportive feedback, clear goals, 
and expectations focusing on learning over grades; doing so will enhance students’ motivation to 
learn. 
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Based upon the idea described above, instructors should work individually with students 
who perform poorly on the first exam in setting appropriate goals for the next exam (or, if possible, 
work individually with each student in setting appropriate goals at the beginning of the semester). 
Elliot, McGregor, and Gable (1991) demonstrated that achievement goals are important predictors 
of motivational study strategies and exam performance. Without clear achievement goals, students 
lack clear motivations and study strategies and perform poorly. 

Therefore, instructors may ask students questions such as: “ What is your goal for the next 
exam?'’’ “Why did you set that goal?" “How are you going to reach your goal?" “Why do you think 
that you can reach that goal?" After the goals are set, instructors need to monitor students and 
provide them with feedback (such as office hour conversations) from time to time and talk with 
them individually to ensure that they are still committed to their goals and being conscientious 
about their work, and answer their questions. As Trautwein, Ludtke, Roberts, Schnyder, and Niggli 
(2009) noted, conscientiousness and competence beliefs are significant indicators of academic 
effort and achievement. For those students who abandon their goals, instructors need to determine 
the reasons for decreased commitment and aid them in modifying or resetting goals to make them 
more appropriate to their level of effort/ability. Note that feedback is necessary in assessing 
whether students’ goals are still effective and ensuring their continued commitment to them. 
Lacking feedback, students may be unaware of their progression or regression, making it more 
difficult for them to determine the level of effort required to effectively achieve their goals. After 
each exam, instructors still need to talk to those students individually. Students should be 
congratulated on reaching their goals; instructors should then work with them individually in 
setting new goals for the next exam. Students who did not attain their goals should receive 
assistance in identifying reasons and developing possible solutions. More importantly, instructors 
need to encourage and work further with them in setting more appropriate new goals for the next 
exam. 

In following this strategy, students (especially lower-performing students) will believe that 
professors care about and are concerned for them and want them to learn. In addition, students will 
comfortably ask questions of, and seek guidance from, their professors. As a result, students may 
develop personal connections with their professors and hence strengthen rapport between students 
and faculty. Engaging in this course of action will enhance students’ motivation to learn and 
confidence in utilizing faculty as a learning resource. More importantly, a strong rapport between 
students and faculty will also simultaneously increase the benefits of studying for a class. 

In addition to the goal-setting strategy, professors may also provide active learning 
exercises, such as the use of game-play in the classroom, to promote students’ motivation to learn 
and encourage students’ participation (e.g., Mahmoud and Tanni, 2014; Lin and Dunphy, 2013). 
As noted by Lin and Dunphy (2013), game-play learning may serve as an important pedagogical 
method. The intent of game-play is to promote students’ motivation to learn/interest in learning 
models and help them easily leam abstract and complicated concepts. 

In short, while it may not be possible for students to reduce their work hours, students’ 
motivation to leam may improve and hence their in-class effort may, too. They may both 
frequently attend classes and study regularly outside the classroom. As a result, students will 
eventually achieve their goals and receive better grades (say, A or B grades), indicating that they 
are making progress in gaining knowledge, and that the relationship between in-class and out-of- 
class efforts is now complementary. 

Limitations 
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This research has two limitations—(1) quality of effort; and (2) time spent for studying—that could 
possibly result in some potential errors. We illustrate these two limitations below. 

Quality of Effort 

The effort used in this study is “quantity of effort” rather than “quality of effort”. While quantity 
of effort can be simply measured by the amount of time, quality of effort cannot be. This is because 
“amount of time” is not a strong predictor of learning outcomes due to the way time is being used, 
whether sitting in the classroom or in a library studying. Plant, Ericsson, Hill, and Asberg (2005) 
demonstrated that the amount of time invested by college students in a class cannot be a good 
predictor of academic performance. In a classroom, some students fall asleep, spend the hour 
texting, play on their computers, or otherwise completely check out, etc. 

We understand the disadvantage of using the amount of time to measure effort, but it is a 
challenge to find an accurate proxy to measure “quality of effort” because “quality of time” cannot 
be simply observed and measured. It is not an easy task for the instructor simultaneously to observe 
and record who is concentrating and who is not, while focusing on teaching. In addition, it is even 
more difficult for an instructor to measure a student’s quality of effort outside the classroom. 

Although “quality of effort” is needed for the study, it cannot be easily observed and 
measured, creating a significant challenge in data collection. To collect such data, we will have to 
develop a strategy that maintains the confidentiality and anonymity of student data and also avoids 
any possibility of negativity. These will be left to future research on this issue. 

Time Spent for Studying 

In addition to the quality of effort limitation, time spent studying is another limitation in this study. 
Time spent studying outside the classroom cannot be observed by the instructor—thus, students’ 
self-reports have to be adopted. For that reason, it is possible that some students may deliberately 
upward-bias self-reporting in this context, which would lead to an error. Fortunately, the regression 
model used in this paper requires differencing data that may either eliminate the error or minimize 
its significance. 

Conclusions 

In this study we explored and discussed in depth an important research question in higher education 
(Do in-class and out-of-class efforts trade off?). We used an empirical model (the complementarity 
formation) to test the trade-off hypothesis between these two efforts. In light of the empirical 
results, three major findings are offered: 

1. For many students, in-class and out-of-class efforts are significantly related and may 
substitute for each other. 

2. For lower-performing students, in-class effort and out-of-class effort are more likely to 
substitute for each other. 

3. Gender is not a factor when ascertaining whether in-class and out-of-class efforts may 
substitute for each other. 

Moreover, we clarified the possible reasons for the occurrence of the trade-off relationship 
in the lower-performing student group and looked at why this relationship could harm higher 
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education. We suggested that instructors work individually with students in setting appropriate 
goals for each exam and frequently offer students feedback. Our suggestion is based on goal¬ 
setting theory 10 which has been widely and successfully used in the workplace as a means of 
improving and sustaining work performance (e.g., Milliman, Zawacki, Chulz, Wiggins, and 
Norman, 1995) and also has been applied in environments in which students are pursuing 
knowledge, such as with English-as-a-Foreign-Language (EFF) and English-as-a-Second- 
Language (ESL) programs (e.g., Jackson, 1999, Cooper, Horn, and Strahan, 2005; Kitsantas, 
2004). Following this strategy may strengthen rapport between students and faculty, thereby 
enhancing students’ motivation to learn and confidence in utilizing faculty as a learning resource. 
In addition to the strategy of goal-setting, we also suggested the strategy of game-play in the 
classroom to promote students’ motivation to learn and encourage students’ participation. 

In summary, the main contribution of this study to higher education is our verification of 
an important fact: a trade-off relationship may exist between in-class and out-of-class efforts for 
many lower-performing students. Verifying this fact offers educators new ways to understand 
student behavior and use that understanding in assisting them to achieve success. In addition, this 
study may also be used to explain why online classes become so popular to many students, 
especially those who are employed, since their employment schedules may sometimes conflict 
with class schedules or they have less time for travel to campus and for rest. For that reason, they 
would prefer to study by themselves outside the classroom rather than sitting inside the classroom 
to learn lectures from instructors. Therefore, ways in which to apply this research to the topic of 
online classes may be an interesting topic for a future study. 
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